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Abstract — We consider utility maximization in networks where 
the sources do not employ flow control and may consequently 
overload the network. In the absence of flow control at the 
sources, some packets will inevitably have to be dropped when the 
network is in overload. To that end, we first develop a distributed, 
threshold-based packet dropping policy that maximizes the 
weighted sum throughput. Next, we consider utility maximization 
and develop a receiver-based flow control scheme that, when 
combined with threshold-based packet dropping, achieves the 
optimal utility. The flow control scheme creates virtual queues at 
the receivers as a push-back mechanism to optimize the amount 
of data delivered to the destinations via back-pressure routing. 
A novel feature of our scheme is that a utility function can 
be assigned to a collection of flows, generalizing the traditional 
approach of optimizing per-flow utilities. Our control policies 
use finite-buffer queues and are independent of arrival statistics. 
Their near-optimal performance is proved and further supported 
by simulation results. 



I. Introduction 

The idea behind flow control in data networks is to regulate 
the source rates in order to prevent network overload, and 
provide fair allocation of resources. In recent years, extensive 
research has been devoted to the problem of network util- 
ity maximization, with the objective of optimizing network 
resource allocation through a combination of source-based 
flow control, routing, and scheduling. The common theme 
is to assign a utility, as a function of the source rate, to a 
flow specified by a source-destination pair, and formulate an 
optimization problem that maximizes the sum of utilities (e.g., 
see (T]-||9)). The optimal network control policy is revealed 
as the algorithm that solves the utility maximization problem. 

Source-based flow control implicitly requires all sources 
to react properly to congestion signals such as packet loss 
or delay. Congestion-insensitive traffic, however, is pervasive 
in modern networks. UDP-based applications such as video 
streaming or VoIP are increasingly popular and do not re- 
spond to congestion. Moreover, greedy or malicious users can 
inject excessive data into the network to either boost self- 
performance or bring down high-profile websites. In these 
circumstances, the network can be temporarily overloaded and 
congestion-aware applications, e.g., TCP-based traffic, may be 
adversely affected or even starved. In this context, source- 
based flow control may not be adequate. 

This work was supported by DTRA grants HDTRA 1-07- 1-0004 and 
HDTRA-09-1-005, ARO Muri grant number W91 1NF-08-1-0238, and NSF 
grant CNS-1 116209. 



There are other scenarios in which optimizing the source 
rates of data flows on a per-flow basis is ineffective. The 
Internet nowadays is vulnerable to distributed denial of service 
(DDoS) attacks 1 10 1, fTT) , in which an attacker creates a large 
number of flows with different source addresses to overwhelm 
prominent websites. Similarly, in a multi-player online game, 
thousands of users require continuous access to the game 
servers. There are also occasions in which a large number of 
users seek access to a website that broadcasts live events (12); 



this is the so-c ailed flash crowd phenomenon 1 13 1. In these sit- 
uations, source-based flow control is ineffective because each 
individual flow uses little bandwidth, but their aggregate traffic 
can lead to severe link congestion near the service provider, 
and may starve other users in the network. A flow control 
scheme that can optimize a utility assigned to a collection of 
flows as opposed to optimizing the sum of per-flow utilities 
can be used to cope with such scenarios. Moreover, in order 
to cope with uncooperative flows, it is necessary to relocate 
the flow control functionality from untmsted network hosts to 
more secure ones, such as the web servers that provide service 
at the receiver end. 

In this paper, we develop such receiver-based flow control 
policies using tools from stochastic network optimization 1 14 1, 
p5j . Our main contributions are three-fold. First, we formulate 
a utility maximization problem that can assign utilities to an 
aggregate of flows, of which the usual per-flow-based utility 
maximization is a special case. Second, given an arbitrary 
arrival rate matrix (possibly outside the network's stability 
region), we characterize the corresponding achievable through- 
put region in terms of queue overflow rates. Third, using 
a novel decomposition of the utility functions, we design 
a network control policy consisting of: (i) a set of flow 
controllers at the receivers; (ii) packet dropping mechanism at 
internal nodes; and (iii) back-pressure routing at intermediate 
nodes. The receiver-based flow controllers adjust throughput 
by modifying the differential backlogs between the receivers 
and their neighboring nodes — a small (or negative) differential 
backlog is regarded as a push-back mechanism to slow down 
data delivery to the receiver. To deal with data that cannot 
be delivered due to network overload, we design a threshold- 
based packet dropping mechanism that discards data whenever 
queues grow beyond certain thresholds. Surprisingly, we show 
that our threshold-based packet dropping scheme, without the 
use of flow control, is sufficient to maximize the weighted 
sum throughput. Moreover, the combined flow control and 
packet dropping mechanism has the following properties: (i) 
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It works with finite-size buffers, (ii) It is nearly utility-optimal 
(throughput-optimal as a special case) and the performance 
gap from the optimal utility goes to zero as buffer sizes 
increase, (iii) It does not require the knowledge of arrival rates 
and therefore is robust to time-varying arrival rates that can 
go far beyond the network's stability region. In addition, our 
control policy can be implemented only in parts of the network 
that include the receivers, treating the rest of the network as 
exogenous data sources (see Fig. [T]for an example), and thus 
might be an attractive flow control solution for web servers. 
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Fig. 1. Our receiver-based policy can be implemented in the whole network 
on the left, or implemented only at nodes A, B, and R on the right, where 
R is the only receiver. The rest of the network on the right may be controlled 
by another network operator or follow a different network control scheme. 

There has been a significant amount of research in the 
general area of stochastic network control. Utility-optimal 
policies that combine source-end flow control with back- 
pressure routing have been studied in |6|-[9| (and references 
therein). These policies optimize per-flow utilities and require 
infinite-capacity buffers. However, they are not robust in the 
face of uncooperative users who may not adhere to the flow 
control scheme. A closely related problem to that studied in 
this paper is that of characterizing the queue overflow rates in 
lossless networks in overload. In a single-commodity network, 
a back-pressure policy is shown to achieve the most balanced 
queue overflow rates fl6) , and controlling queue growth rates 
using the max-weight policy is discussed in p7) . The queue 
growth rates in networks under max-weight and a-fairness 
policies are analyzed in fl8) , fl9) . We finally note that the 
importance of controlling an aggregate of data flows has been 
addressed in fT3) , and rate-limiting mechanisms in front of a 
web server to achieve some notion of max-min fairness against 
DDoS attacks have been proposed in [20|-|22|. 



An outline of the paper is as follows. The network model 
is given in Section [IT] We formulate the utility maximization 
problem and characterize the achievable throughput region in 
terms of queue overflow rates in Section III Section IV intro- 



duces a threshold-based packet dropping policy that maximizes 
the weighted sum throughput without the use of flow control. 
Section [V] presents a receiver-based flow control and packet 
dropping policy that solves the general utility maximization 
problem. Simulation results that demonstrate the near-optimal 
performance of our policies are given in Sections IV and [V] 



II. Network Model 

We consider a network with nodes Af = {1,2, ... ,N} 
and directed links C = {(n,m) \ n, m G Af}. Assume 
time is slotted. In every slot, packets randomly arrive at the 
network for service and are categorized into a collection C 



of classes. The definition of a data class is quite flexible 
except that we assume packets in a class c £ C have a 
shared destination d c . For example, each class can simply 
be a flow specified by a source-destination pair. Alternatively, 
computing-on-demand services in the cloud such as Amazon 
(Elastic Compute Cloud; EC2) or Google (App Engine) can 
assign business users to one class and residential users to 
another. Media streaming applications may categorize users 
into classes according to different levels of subscription to the 
service provider. While classification of users/flows in various 
contexts is a subject of significant importance, in this paper we 
assume for simplicity that the class to which a packet belongs 
can be ascertained from information contained in the packet 
(e.g., source/destination address, tag, priority field, etc.). Let 
(i) 6 {0, 1, . . . , Anax} be the number of exogenous class c 
packets arriving at node n in slot t, where A max is a finite 
constant; let A^f\t) = for all t. We assume A„ (t) are 
independent across classes c and nodes n ^ d c , and are i.i.d. 
over slots with mean E[yl„ (£)] = X„ ,. 

In the network, packets are relayed toward the destinations 
via dynamic routing and link rate allocation decisions. Each 
link (n, to) 6 C is used to transmits data from node n to node 
to and has a fixed capacity /i™ (in units of packets/slot)Q 
Under a given control policy, let /inm(t) be the service rate 
allocated to class c data over link (n, m) in slot t. The service 
rates must satisfy the link capacity constraints 

Yj MnmO) < /Ci> for a11 t and all links (n, to). 

cec 

At a node n ^ d c , class c packets that arrive but not 
yet transmitted are stored in a queue; we let Q { n\t) be the 
backlog of class c packets at node n at time t. We assume 
initially (0) = for all n and c. Destinations do not buffer 
packets and we have Qf' (t) — for all c and t. For now, we 
assume every queue Qn (t) has an infinite-capacity buffer; we 
show later that our control policy needs only finite buffers. To 
resolve potential network congestion due to traffic overload, 
a queue Qn\t), after transmitting data to neighboring nodes 
in a slot, discards d„(t) packets from the remaining backlog 
at the end of the slot. The drop rate dn \t) is a function of 
the control policy to be described later and takes values in 
[0,dmax] for some finite d max . The queue Qn\t) evolves over 
slots according to 



Q { n\t+l)< 



(Ql c) (t)-E^W) + -4 c) W 

b 



(1) 



where (•)+ = max(-,0). This inequality is due to the fact 
that endogenous arrivals may be less than the allocated rate 
XL Mem (£) when neighboring nodes do not have sufficient 
packets to send. 

For convenience, we define the maximum transmission rate 

'We focus on wireline networks in this paper for ease of exposition. Our 
results and analysis can be easily generalized to wireless networks or switched 
networks in which link rate allocations are subject to interference constraints. 
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into and out of a node by 



in A \ 

/W = max 2^ 



LL aTL 



Mmax — z2 ^max 

6:(n,6)e£ 



(2) 



Throughout the paper, we use the following assumption. 

Assumption 1. We assume d max > A max + /i}^. 

From ([TJ), the sum A max + ^™ ax is the largest amount of 
data that can arrive at a node in a slot; therefore it is an 
upper bound on the maximum queue overflow rate at any node. 
Assumption [TJ ensures that the maximum packet dropping rate 
dmax is no less than the maximum queue overflow rate, so that 
we can always prevent the queues from blowing up. 

III. Problem Formulation 

We assign to each class c € C a utility function g c {-)- Given 
an (unknown) arrival rate matrix A = (An ), let Aa be the 
set of all achievable throughput vectors (r c ), where r c is the 
aggregate throughput of class c received by the destination d c . 
Note that Aa is a function of A. We seek to design a control 
policy that solves the global utility maximization problem 



maximize 



,9c(r c ) 
subject to (r c ) G A\, 



(3) 



(4) 



where the region Aa is presented later in Lemma[TJ We assume 
all g c (-) functions are concave, increasing, and continuously 
differentiable. For ease of exposition, we also assume the func- 
tions <7c(0 nave bounded derivatives such that < m c 
for all x > 0, where m c are finite constants]^] 

As an example, consider the tree network in Fig. [2] that 
serves three classes of traffic destined for node R. Class 1 



class 1 



class 2 class 1 class 3 





Fig. 2. A tree network with three classes of traffic. 

data originates from two different sources A and C, and may 
represent the collection of users located in different parts of 
the network sending or requesting information from node R. 
If class 1 traffic is congestion-insensitive and overloads the 
network, without proper flow control class 2 and 3 will be 
starved due to the presence of class 1. A utility maximization 
problem here is to solve 

maximize gi{r 1A + r ic ) + #2(725) + 93^30) (5) 
subject to (r 1A ,nc,r2B,r 3 D) feasible, (6) 

2 Utility functions g c (%) that have unbounded derivatives as x — > can 
be approximated by those with bounded derivatives. For example, we may 
approximate the proportional fairness function log(x) by log(x + £) for some 
small £ > 0. 



where t\a denotes the throughput of class 1 data originating 
from A; r\c, r%B, and r^r> are defined similarly. Note that this 
utility maximization |5j-([6j is very different from, and gener- 
alizes, the traditional per-fiow-based utility maximization. 

A. Achievable Throughput Region 

The next lemma characterizes the set Aa of all achievable 
throughput vectors in |4j. 

Lemma 1. Under i.i.d. arrival processes with an arrival rate 
matrix A = (An ), let Aa be the closure of the set of all 
achievable throughput vectors (r c ). Then (r c ) G Aa if and 
only if there exist flow variables > | c € C, (a, b) € £} 
and queue overflow variables > | c G C, 11^ d c } such 
that 

Al c) + £ = <?i c) + £ Vc G C, n jk d c , (7) 



£/$</& vm)g£, 



— ^ ^ f 1 ad c 



VcgC. 



(8) 



(9) 



In other words, 

Aa = { (r c ) I 0-© hold and > 0, gC«0 > } . 

Proof of Lemma |7| See Appendix [A] ■ 
In Lemma [TJ equation (|7]i is the flow conservation constraint 
stating that the total flow rate of a class into a node is equal 
to the flow rate out of the node plus the queue overflow rate. 
Equation |8]l is the link capacity constraint. The equality in |9]) 
shows that the throughput of a class is equal to the sum of 
exogenous arrival rates less the queue overflow rates. Lemma[T] 
is closely related to the network capacity region A defined 
in terms of admissible arrival rates (see Definition [TJ; their 
relationship is shown in the next corollary. 



Definition 1 (Theorem 1, |23|). The capacity region A of the 



network is the set of all arrival rate matrices (An ) for which 
there exists nonnegative flow variables ff£ such that 

a! c ) + \ /■;:;: > r, 1 . Vcec, n^d c 



V f (c) < 

/ j j an — 



V f (c) 

/ , Jnb ' 
b 



f^f<A, V(o,6)g£. 



(10) 



(ID 



Corollary 1. An arrival rate matrix (Al c ^) lies in the network 

capacity region A if there exist flow variables fj$ > 

such that flow conservation constraints d7]l and link capacity 

, . { c \ ' — ' 

constraints (|8]l hold with g„ = for all n and c. 

Corollary [T| shows that (Xn^) is achievable if and only if 
there exists a control policy yielding zero queue overflow rates. 
In this case the throughput of class c is r c = J2 n ■ 

We remark that the solution to the utility maximization ([3]l- 
Q activates the linear constraints |9]); thus the problem ([3j-<|4j) 
is equivalent to 



maximize 



£.9c(r c ) 



(12) 



cec 
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subject to r c = VcGC (13) 



{7} and ((HJl hold (14) 

fj$>0, <?i c) >0, V(a,6)eA Vn,c. (15) 

Let (r*) be the optimal throughput vector that solves ( fT2| i- 
(Tf5). If the arrival rate matrix (An ) is in the network capacity 
region A, the optimal throughput is r* = J2 n ^« from 
Corollary 111 Otherwise, we have r* = Xm<^« — En?" ' 
where is the optimal queue overflow rate. 

B. Features of Network Control 

Our control policy that solves (fT2]>-(fT5]l has two main fea- 
tures. First, we have a packet dropping mechanism discarding 
data from the network when queues build up. An observation 
here is that, in order to optimize throughput and keep the 
network stable, we should drive the packet dropping rate to 
be equal to the optimal queue overflow rate. Second, we need a 
flow controller driving the throughput vector toward the utility- 
optimal point. To convert the control objective ( fT2] i into these 
two control features, we define, for each class c G C, a utility 
function h c (-) related to g c () as 



M r c) = g c (r c ) - c r c 



(16) 



where 8 C > are control parameters to be decided later. 
Using ( fT3j ), we have 



g c {r c ) = h c {r c )+e c [Y, X n ) -Y, 



Since An are unknown constants, maximizing J2ctc 9c( r c) 
is the same as maximizing 



E [mo 



cec 



This equivalent objective ( fT7] i can be optimized by jointly 
maximizing the new utility J^cec ^c( r c) at me receivers 
and minimizing the weighted queue overflow rates (i.e., the 

weighted packet dropping rates) X) c gc ® c ^ at eacn n °de n. 

Optimizing the throughput at the receivers amounts to con- 
trolling the amount of data actually delivered. This is difficult 
because the data available to the receivers at their upstream 
nodes is highly correlated with control decisions taken in the 
rest of the network. Optimizing the packet dropping rates 
depends on the data available at each network node, which 
has similar difficulties. To get around these difficulties, we 

(c) 

introduce auxiliary control variables tp n ' > and v c > and 
consider the optimization problem 



maximize 



subject to 



E 



[/lc(O-0cE 



(18) 



VcgC, (19) 
VcgC, n^d c . (20) 
([13]>-(03]> hold. (21) 

This is an equivalent problem to (|T2|>-(|T3]>. The constraints ( fT9| ) 
and ( p0| > can be enforced by stabilizing virtual queues that will 
appear in our control policy. The new control variables v c and 



(c) 

ip n to be optimized can now be chosen freely unconstrained 
by past control actions in the network. Introducing auxiliary 
variables and setting up virtual queues are at the heart of using 
Lyapunov drift theory to solve network optimization problems. 

IV. Maximizing the Weighted Sum Throughput 

For ease of exposition, we first consider the special case of 
maximizing the weighted sum throughput in the network. For 
each class c G C, we let g c (r c ) — &c r c for some a c > 0. We 
present a threshold-based packet dropping policy that, together 
with back-pressure routing, solves this problem. Surprisingly, 
flow control is not needed here. This is because maximizing 
the weighted sum throughput is equivalent to minimizing the 
weighted packet dropping rate. Indeed, choosing 9 C = a c 
in (TT31, we have h c = for all classes c, under which 
maximizing the equivalent objective ( ff7] i is the same as 
minimizing ^ n C 6 C gl c \ In the next section, we will combine 
the threshold-based packet dropping policy with receiver-based 
flow control to solve the general utility maximization problem. 

A. Control Policy 

To optimize packet dropping rates, we set up a drop queue 
Dn\t) associated with each queue Qn\t). The packets that 
are dropped from Qn^ (t) in a slot, denoted by dffl (t), are first 
stored in (t) for eventual deletion. From ([T]), we have 



<#°(*) 



Note that the quantity cfn\t) is the actual packets dropped 
from Qn (t), which is strictly less than the allocated drop rate 
d^n 1 (t) if queue Q„ (t) does not have sufficient data. Packets 



(22) 



(17) are permanently deleted from D$(t) at the rate of ^'(t) € 



Mi 



[0, d max ] in slot t. The queue D„ (t) evolves according to 



D[?{t + 1) = [D^\t) - ^\t)} + + dtfit). 



(23) 



Assume initially D„ , (0) = V6 C = Var for all n and c, 
where V > is a control parameter^! If queue Dn\t) 
is stabilized, then minimizing the service rate of Dn\t) 
effectively minimizes the time average of dropped packets at 
Q ( n\t). We propose the following policy. 

Overload Resilient Algorithm (OR A) 

Parameter Selection: Choose 9 C = a c for all classes c G C, 
where g c (x) — a c x. Choose a parameter V > 0. 

Backpressure Routing: Over each link I — (n, m) G C, let 
C; be the subset of classes that have access to link I. Compute 
the differential backlog V^/ c) (i) = Q { n ] (t) - Q { ^{t) for each 
class c G Ci, where Q^\t) — at the receiver d c . Define 



W l (c, *(t) = maxW l (c, (t), 
c*(t) = argmax ceC! wf\t). 

It suffices to assume Z)l c) (0) to be finite. Our choice of d£ c) (0) = V9 C 
avoids unnecessaiy packet dropping in the initial phase of the system. 
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We allocate the service rates 



_ J ^max 



if VF/ C) *(t) >o, 
if VF, (c) *(t) < 0. 



Let fJnm(t) = for all classes c = C; \ {c ; *(£)}. 

Packet Dropping: At queue Q^(i), allocate the packet 
dropping rate dn (t) (see ([l}) according to 

, (c)m _ Jdmax ifQi C) (t)>^i C) (t), 

" (j \0 ifQ( c) (t)<^ c) (<), 

where d max > is a constant chosen to satisfy Assumption [T] 
At the drop queue D n c \t), allocate its service rate tpn (t) 
according to 

'd max if D^\t) >V9 C , 



if D { n\t) < ve c . 

(c), 



Queue Update: Update queues Q„ (t) according to ([T| and 
update queues D„\t) according to ([22]>-([23]> in every slot. 

The packet dropping subroutine in this policy is threshold- 
based. The ORA policy uses local queueing information and 
does not require the knowledge of exogenous arrival rates. 
It is notable that network overload is autonomously resolved 
by each node making local decisions of routing and packet 
dropping. 



sum throughput satisfying 



c£C 



c£C 



B 
V' 



(26) 



where (r*) is the optimal throughput vector that solves ( |T2| i- 
( p"5j ) under the linear objective function J^cec ° c r c, V > is 
a control parameter, and B is a finite constant defined as 



i 2 1 



B 4 \M\ \C\ [OC + d,nax) 2 + (A max + ^J 2 + 2di 

where |.4| denotes the cardinality of a set A. 

We omit the proof of Theorem [T] because it is similar to 
that of Theorem [2] presented later in the general case of utility 
maximization. From ( p6| ), the ORA policy yields near-optimal 
performance by choosing the parameter V sufficiently large. 
Correspondingly, a large V implies a large buffer size of 
— V0 4- 2d 

As shown in Corollary 111 if the arrival rate matrix (A^) lies 
in the network capacity region A, then the optimal throughput 
for class c is r* = J2 n anc l reduces to 

B 
V 



^« £ ^>E« c (E A i c) ) 

cec cec n 



That we can choose V arbitrarily large leads to the next 
corollary. 

Corollary 2. The ORA policy is (close to) throughput optimal. 



B. Performance of the ORA Policy 

Lemma 2 (Deterministic Bound for Queues). For each class 
c € C, define the constants 

= V°c + 4ax, = V6 C + 2d max . (24) 

In the ORA policy, queues Qn (t) and Dn \t) are determin- 
istically bounded by 

Q { n\t)<Qt± D${t)<D& 



C. Simulation of the ORA Policy 

We conduct simulations for the ORA policy in the network 
shown in Fig. [3] The directed links (A, B) and (B, C) have 

I cla ss 3 1 1 class 1 j 




<5) 



class 2 



for all t, C, and 71 ^ d c . Fig. 3. A 3-node network with three classes of traffic. 



In addition, we have D^f 1 (i) > V 6 C — d max for all n, c, and t. 

Proof: See Appendix [B] ■ 
In Lemma [2] the value of Qmax is the finite buffer size suffi- 
cient at queue Qn\t). The parameter V controls when queue 
Qn\t) starts dropping packets. Indeed, due to Dn\t) > 
V9 C — dmax, the ORA policy discards packets from Qn (t) 
only if Q^ (t) > V9 C — d max . The quantity V9 C — d max is a 
controllable threshold beyond which we say queue Q^ (t) is 
overloaded and should start dropping packets. As we see next, 
the performance of the ORA policy approaches optimality as 
the buffer sizes increase. 



Theorem 1. Define the limiting throughput of class c as 



(25) 



where (Jf] (r) denotes the class c packets received by node d c 
over link (a, d c ). The ORA policy yields the limiting weighted 



the capacity of 1 packet/slot. There are three classes of traffic 
to be served; for example, class 1 data arrives at node B and 
is destined for node C. Classes 1 and 2 compete for service 
over (B, C); classes 2 and 3 compete for service over (A, B). 
Each simulation below is run over 10 6 slots. 

1 ) Fixed arrival rates: In each class, we assume a Bernoulli 
arrival process whereby 20 packets arrive to the network in 
a slot with probability 0.1, and no packets arrive otherwise. 
The arrival rate of each class is 2 packets/slot, which clearly 
overloads the network. 

Let r c be the throughput of class c. Consider the objective 
of maximizing the weighted sum throughput 3 r\ + 2 r 2 + r 3 ; 
the weights are rewards obtained by serving a packet in a 
class. The optimal solution is: (i) Always serve class 1 at node 
B because it yields better rewards than serving class 2. (ii) 
Always serve class 3 at node A — although class 2 has better 
rewards than class 3, it does not make sense to serve class 2 at 
A only to be dropped later at B. The optimal throughput vector 
is therefore (1, 0, 1). Consider another objective of maximizing 



TABLE I 

The throughput performance of the ORA policy under fixed 

ARRIVAL RATES. 

(a) Maximizing 3 n + 2 T2 + 7"3 (b) Maximizing 3 n + 5 r2 + r3 
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TABLE II 

The throughput performance of the ORA policy under 

TIME-VARYING ARRIVAL RATES. 



time interval 


throughput 


optimal 




in this interval 


value 


[0,3- 10 5 ) 


(.797,.097, .771) 


(.8,-1,-8) 


[3- 10 5 ,6- 10 5 ) 


(.001,.998,0) 


(0,1,0) 


[6- 10 5 ,10 6 ) 


(.798,-102,-772) 


(.8,-1,-8) 



(a) Queue Q% ] (t) 









'.Man 







(c) Queue Q ( ? } (t) 



liUlUuJM 



(b) Queue Q l g } (t) 




(d) Queue Q ( ^(t) 



Fig. 4. The queue processes under the ORA policy with time-varying arrival 
rates that temporarily overload the network. 



3 r\ + 5 T2 + ra. Here, class 2 has a reward that is better than 
the sum of rewards of the other two classes. Thus both nodes 
A and B should always serve class 2; the optimal throughput 
vector is (0, 1,0). Table [I] shows the near-optimal performance 
of the ORA policy in both cases as V increases. 

2) Time-varying arrival rates: We show that the ORA 
policy is robust to time-varying arrival rates. Suppose class 
1 and 3 have a fixed arrival rate of 0.8 packets/slot. The 
arrival rate of class 2 is 2 packets/slot in the interval T = 
[3x 10 5 , 6x 10 5 ) and is 0.1 packets/slot elsewhere. We consider 
the objective of maximizing 3ri + 5r2 + r^. The network is 
temporarily overloaded in the interval T; the optimal time- 
average throughput in T is (0, 1, 0) as explained in the above 
case. The network is underloaded in the interval [0, 10 6 ) \ T, 
in which the optimal throughput is (0.8,0.1,0.8). 

We use the following parameters here: V = 100, A m . dx — 
20, cUx = 4nax + /C,x = 21, and (0i,0 2 ,0 3 ) = (3,5,1). 
Table shows the near-optimal throughput performance of 
the ORA policy. Figure [4] shows the sample paths of the 
queue processes Q%\t), Qg\t), Q^\t), and Q^\t) in the 
simulation. Clearly the queues suddenly build up when the 
network enters the overload interval T, but the backlogs are 

(c) 

kept close to the upper bound QJnax = V6 C + 2d max without 
blowing up. 

V. Utility-Optimal Control 

We solve the general utility maximization problem ([3]>-(|4j> 
with a network control policy very similar to the ORA policy 
in the previous section except for an additional flow control 
mechanism. 

A. Virtue Queue 



variables and r c is the throughput of class c. To enforce the 
constraint r c = v c , we construct a virtual queue Z c (t) in which 
r c is the virtual arrival rate and v c is the time-average service 
rate. Let (t) be the class c packets received by node d c 
over link (a,d c ); we have 



/&(*) = 



In Section III-B we formulate the equivalent optimization 
problem (fT8|-(21 1 that involves maximizing J2 c ec h c {v c ) sub- 
ject to r c — v c for all classes c, where v c are auxiliary control 



The arrivals to the virtual queue Z c {t) in a slot are the total 
class c packets delivered in that slot, namely, ^ Jy^J (t). Let 
v c (t) be the allocated virtual service rate at Z c (t) in slot t. The 
virtual queue Z c (t) is located at the receiver d c and evolves 
according to 

Z c (t + 1) = [Z c (t) - v c {t)] + + 5>i c J c (27) 

a 

Assume initially Z c (0) — for all classes c. It is well known 
that if queue Z c (t) is stable then r c < v c . But we are interested 
in the stronger relationship that stabilizing Z c (t) leads to r c = 
v c . To make it happen, it suffices to guarantee that queue Z c (t) 
wastes as few service opportunities as possible, so that the 
time-average allocated service rate v c is approximately equal 
to the throughput out of queue Z c (t). For this, we need two 
conditions: 

1) The queues Z c (t) usually have more than enough (vir- 
tual) data to serve. 

2) When Z c (t) does not have sufficient data, the allocated 
service rate v c {t) is made arbitrarily small. 

To attain the first condition, we use an exponential-type Lya- 
punov function that bounds the virtual backlog process Z c (t) 
away from zero (and centers it around a parameter Q > 0). 
The second condition is attained by a proper choice of the 
parameters 9 C to be decided later. 
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B. Control Policy 

The following policy, constructed in Appendix [C] a nd [D] 
solves the general utility maximization problem ||3)-Q. 

Utility-Optimal Overload-Resilient Algorithm (UORA) 

Parameter Selection: Choose positive parameters v max , w, 
V, Q, and {9 C , c e C} to be discussed shortly. Assume initially 
Q&°(0) = Z c (0) = and D^(0) = V6 C . 

Packet Dropping: Same as the ORA policy. 

Backpressure Routing: Same as the ORA policy, except 
that the differential backlog over each link I = (a,d c ) E C 
connected to a receiver d r is modified as: 



wf\t) = Q^(t)-Q£(t), 
where we abuse the notation by redefining 



,(c) 



(<) 



' we w{Z c {t)-Q) tf z c (t)>Q 

_ we w(Q-Z c (t)) if Zc (t)<Q 



(28) 



(29) 



for all classes c. The exponential form of (t) is a result 
of using exponential-type Lyapunov functions. We emphasize 
that here (t) has nothing to do with real data buffered at 
the receivers (which must be zero); it is just a function of the 
virtual queue backlog Z c (t) that gives us the "desired force" 
in the form of differential backlog in |28) to pull or push- 
back data in the network. Thus, unlike standard back-pressure 
routing that has Q f (t) = 0, here we use Q f (t) as part of 
the receiver-based flow control mechanism. 

Receiver-Based Flow Control: At a destination d c , choose 
the virtual service rate v c (t) of queue Z c (t) as the solution to 

maximize Vh c (v c (t)) + v c (t) Q^]{t) (30) 
subject to < v c (t) < ^ max (31) 

where h c (x) — g c (x) — 9 c x. 

Queue Update: Update queues Qn\t), Dn\t), and Z c {t) 
according to ([TJ, ( |23] l, and ( p7j ), respectively, in every slot. 



C. Choice of Parameters 

We briefly discuss how the parameters in the UORA policy 
are chosen. Let e > be a small constant which affects the 
performance of the UORA policy (cf. ((33])). In ([30])-([3l}, we 
need the parameter v mdLX to satisfy i^ max > max ce c r* + e/2, 
where (r*) is solution to the utility maximization (one 
feasible choice of v max is the sum of capacities of all links 
connected to the receivers plus e/2). This choice of f max 
ensures that queue Z c (t) can be stabilized when its virtual 
arrival rate is the optimal throughput r*. Due to technical 
reasons, we define (5 max = max[z/ max , ^™ ax ] and choose the 
parameter 



max 



e/<5„, 



in p9| ). The parameter Q (see ( |29| l) is used to bound the queues 
Z c {t) away from zero and center them around Q; for technical 
reasons, we need Q > is max . The parameters 8 C are chosen to 



satisfy h' c (x) — g' c (x) — 9 c < for all x > e. This ensures that, 
when Z c (t) < Q, its virtual service rate v c {t) as the solution 
to (|3Ql)-pT|) is less than or equal to e, attaining the second 
condition mentioned in Section V-A to equalize the arrival rate 
and the time-average service rate of the virtual queue Z c (t) 
(see Lemma [6] in Appendix [F]). The parameter V captures the 
tradeoff between utility and buffer sizes to be shown shortly 
and should be chosen large; for technical reasons, we need V 
to satisfy V9 C + 2d max > w. 

D. Performance Analysis 

Lemma 3. In the UORA policy, queues Q„ (t), Dn \t), and 
Z c (t) are deterministically bounded by 

Q ( n ] (t) < D\f> (t) < 4, z&) < 

for all t, c, and n, where Q max and Z3m ax are defined in p4| ) 
and Z max is defined as 

rylc) A ^| . 1 , ( + 2(i max \ ■ 

= Q + - log + AVax- (32) 

w \ w J 

Proof of Lemma^ See Appendix [E] ■ 

Theorem 2. The UORA policy yields the limiting utility that 
satisfies 

£s c (^) > £<?c(r*) - ^ -yBmc + JJ, (33) 

c c c 

where fc is defined in ( p5| , (r* ) is the throughput vector that 
solves the utility maximization problem ([3])-(j4j>, and B\ is a 
finite constant (defined in dSUJl). 



Proof: See Appendix [F] ■ 
Theorem[2]shows that the performance gap from the optimal 
utility can be made arbitrarily small by choosing a large V and 
a small e. The performance tradeoff of choosing a large V is 
again on the required finite buffer size Qmax = + 2d max . 

E. Simulation of the UORA Policy 
We conduct two sets of simulations. 

1) On the 3-node network in Fig. [ij The goal is to provide 
proportional fairness to the three classes of traffic; equivalently 
we maximize the objective function log(ri)+log(r2)+log(r3). 
Each directed link (A, B) and (B, C) has the capacity of one 
packet per slot. The arrival process for each class is that, in 
every slot, 20 packets arrive to the network with probability 
0.1 and zero packets arrive otherwise. The arrival rate vector is 
(2, 2, 2), which overloads the network. In this network setting, 
due to symmetry the optimal throughput for class 1 is equal 
to that of class 3, which is the solution to the simple convex 
program 

maximize: 2\og{x) + log(l — x), subject to: < x < 1. 

The optimal throughput vector is (2/3,1/3,2/3) and the 
optimal utility is —1.91. 

As explained in Section |V-C| we choose the parameters of 
the UORA policy as follows. Let e = 0.1. To satisfy 9 C > 1/x 
for all x > e, we choose 9 C = 1/e = 10 for all classes c. 
The value of /i™ ax in the 3-node network is one. The optimal 
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TABLE III 

The throughput performance of the UORA policy in the 3-node 

NETWORK 



TABLE V 

The throughput performance of the UORA policy in the tree 

NETWORK 



V 


ri 


T2 


r3 


E log(r c ) 


V 


ri 


T2 




10 


.522 


.478 


.522 


-2.038 


10 


.200 


.100 


.100 


20 


.585 


.415 


.585 


-1.952 


20 


.364 


.206 


.205 


50 


.631 


.369 


.631 


-1.918 


30 


.661 


.650 


.651 


100 


.648 


.352 


.647 


-1.912 


50 


.667 


.667 


.667 


optimal 


.667 


.333 


.667 


-1.910 


optimal 


.667 


.667 


.667 



TABLE IV 

Maximum backlog in queues under the UORA policy in the 

3-NODE NETWORK 



V 




Qf{t) 


Qf(t) 




o (c) 

^6 max 


10 


140 


97 


137 


137 


142 


20 


237 


187 


240 


236 


242 


50 


539 


441 


538 


540 


542 


100 


1036 


865 


1039 


1039 


1042 



throughput vector satisfies max c r* = 1 and we choose 
^max = 3 (any value of v m3X greater than max c r*+e/2 = 1.05 
works). By definition 5 max = max[i/ max , ^™ ax ] = 3. In the 
arrival processes we have A m . ix = 20. By Assumption [TJ we 
choose d max = A max + ^ ax = 21. Let Q = 1000. 

We simulate the UORA policy for different values of V. 
The simulation time is 10 6 slots. The near-optimal throughput 
performance is given in Table [Ell] Table |IV] shows the max- 
imum backlog in each queue Qh (t) during the simulation. 
Consistent with Lemma [3] the maximum backlog is bounded 



by Oil = V6 C + 2d max - 10V + 42. 

2) On the tree network in Fig. |2j Consider providing max- 
min fairness to the three classes of traffic in Fig. [2] Each 
link has the capacity of one packet per slot. Each one of 
the four arrival processes has 20 packets arriving in a slot 
with probability 0.1 and zero packets arrive otherwise. The 
arrival rates are (2,2,2,2), which overloads the network. The 
optimal throughput for the three classes is easily seen to be 
(2/3, 2/3,2/3), where each flow of class 1 contributes equally 
in that class. 

We approximate max-min fairness by using the a-fairness 
functions g c {x) = x x ~ a j (I— a) with a large value of a = 100. 
The utility maximization becomes: 

maximize — [(r 1A + r ic y" + (r 2 i?) -99 + (r 3D )~ 99 ] 
yy 

subject to {r 1A ,r lc , r 2 B,r^ D ) feasible in Fig. [5J 

where t\a is the throughput of class 1 flow originating from 
node A; the other variables are similarly defined. 



According to Section V-C we choose the parameters of 
the UORA policy as follows. We require 8 C > a; -100 for all 
x > e. For convenience, let us choose 8 C = e = 1 for all 
classes c. The optimal throughput vector satisfies max c r* = 2, 
achieved when the network always serves class 1. We choose 
^max = 4 (any value of ^ max greater than max c r* + e/2 = 2.5 
works). We observe from Fig. [2] that ^™ ax = 2, and we have 



■Jin 



4. We have A„ 



20 in the 



arrival processes and by Assumption [T] we choose rf max = 
A max + aCx = 22. Let Q = 100. 

We simulate the UORA policy for different values of V and 
each simulation takes 10 6 slots. The near-optimal performance 
of the UORA policy is given in Table |V| 

VI. Conclusion 

In this paper we develop a receiver-based flow control and 
an in-network packet dropping strategy to cope with network 
overload. Our scheme is robust to uncooperative users who 
do not employ source-based flow control and malicious users 
that intentionally overload the network. A novel feature of our 
scheme is a receiver-based backpressure/push-back mechanism 
that regulates data flows at the granularity of traffic classes, 
where packets can be classified based on aggregates of data 
flows. This is in contrast to source-based schemes that can only 
differentiate between source-destination pairs. We show that 
when the receiver-based flow control scheme is combined with 
a threshold-based packet dropping policy at internal network 
nodes, optimal utility can be achieved. 
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Appendix A 
Proof of LemmaQ] 

First we show |7|-(|9| are necessary conditions. Given a 
control policy, let F^{t) be the amount of class c packets 
transmitted over link (a, b) in the interval [0, t], and Qn\t) 
be the class c packets queued at node n at time t. From the fact 
that the difference between incoming and outgoing packets at 
a node in [0, t] is equal to the queue backlog at time t, we 
have 



5>! c) (t-) - Q^W + E^ (34) 

r=0 a b 

which holds for all nodes n =^ d c for each class c € C. Taking 
expectation and time average of (34), we obtain 



1, 



-E 



E^c*) 



E 



Q { n\t) 



1, 



-E 



(35) 



L b 



The link capacity constraints lead to 

^E E [^ } (*)] ^> 



v(o,6) e c. 



(36) 



Consider the sequences E[F^\t)]/t and E[Ql c) (<)]/t in- 
dexed by t in (|35|). For each c € C and (a, b) G C, the 



sequence {E[Ff$(t)]/t,t € 1 + } is bounded because the 
capacity of each link is bounded. It follows from ([35} that 
the sequence {E[Q„ (t)]/t, t € Z + } is also bounded. There 
is a subsequence {tk} such that limit points f^jj and qffi exist 
and satisfy, as k — > oo, 

' "fcaM fit VceC, V(a,6)e£, (37) 



— E 

Ie 



Q^itk) 



q (c) 



V e e C , V n 7^ d c 



(38) 



Applying <|37)-<[38) to (|35])-((36]) results in Q and ((S). Define 
the throughput r c of class c as 

1 ^ ■ 



r c = lim inf - 

t— >oo t 



(39) 



a:(a,d c )£C 

The inequality in |9]) follows ((39]) and §37). The equality in |9]) 
results from summing |7]i over n ^ d c . 



To show the converse, it suffices to show that every interior 
point of Aa is achievable. Let (r c ) ce c be an interior point of 
A\, i.e., there exists e € (0,1) such that (r c + e) c ec G A\. 

(c) (c) 

There exist corresponding flow variables f^ b and q„ such 
that 

Ai c) + E f£ = & + E fnb\ E fS * /C, (40) 

a be 

r c + e <E A » c) -E9i c) - (4i) 

n n 

In the flow system (|40]i-(|4T), by removing the subflows that 
contribute to queue overflows, we obtain new reduced flow 
variables \ { n ] and ftf such that < < \ { n ] , < fjtf < 
fit and 



f(c) 
, J a 



^ ^ J an 



Jnb 



/ j J n 
b 



7, Jab H> 



Define 



r (c) A ^(c) 



e < 



1 - 



, V c, Vn 7^ d c 

V(o,6) e C, 
Ai c) , VceC. 



ab 
max' 



(42) 
(43) 
(44) 



e/2 



E„Ai c) j En W -e/2 



From we have J2 n A » ' > e/2 and J2„ An } - e/2 > r, 



It is not difficult to check that r n ' > 0, r n ' 



J2n r " 



(c) 



< Al c) , and 



Combined with (|42|)-(|43|), we obtain 

V f (c) < V f (c) V f (c) < u ab 

/ j J an ^ /_^Jnbi J ab — P"max 



where the first inequality is a strict one. These inequalities 
show that the rate matrix (ri^) is an interior point of the 
network capacity region A in Definition [T] and therefore is 
achievable by a control policy, such as the back-pressure 



policy |23|. Therefore, the aggregate rate vector (r c ), where 



J2 n r " ' ^ S a ^ so achievable. 
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Appendix B 
Proof of Lemma[2] 

We prove Lemma [2] by induction. First we show Dn\t) is 
deterministically bounded. Assume Dn\t) < D^} x for some 
t, which holds at t = because we let L>i c) (0) = V6 C . 
Consider the two cases: 

1) If D ( n\t) < V6 C , then from (|23) we obtain 

D ( :\t + 1) < D^(t) + d^{t) < V0 C + d IWdX = 

where the second inequality uses ( |22] i. 

2) If D ( n \t) > V6 C , then the ORA policy chooses 
tpn (t) — d mdx at queue D^tf* (t) and we have 

D ( n c) (t + 1) < [£>W (t) - d mdx ] + + d m . dx 
<max[DV>(t),d aa ]<D}& l , 

where the last inequality uses the induction assumption. 
From these two cases, we obtain D ( n c) (t + 1) < D^ x . 

Similarly, we show (t) is bounded. Assume Q„ {t) < 



c,n^d c 

^(Zc(t)-Q) + e w(Q-Z c (t 



c,n^d c 



)(c) 
; ma; 



for some <, which holds at i = because we let 



Qn (0) = 0. Consider the two cases: 

1) If Q { n\t) < £>£L then from {[} we get 

< 0^(t)+A +u in < +d — 

where the second inequality uses Assumption [TJ 

2) If Q { n\t) > Anax > I>4 c) (i), the ORA policy chooses 
dn\t) = d mdx at queue Qn\t) and we have 

Qtf(t + 1) < g^>(t) - d ma x + Anax + /Cx 

<gl c) w<QLi, 

where the third inequality uses induction assumption. 
We conclude that Q { n\t + 1) < Q ( mL 

Finally, we show D„ (t) > V9 c — d mdx for all slots. Assume 
this is true at some time t; this holds when t = because we 
assume Dn\o) = V9 C . Consider the two cases: 

1) If D ( n\t) < V9 C , the ORA policy chooses ip ( n\t) = 
at queue Dn\t) and we have 

Di c) (t + 1) > £>£> (t) > ve c - d nmx 



The last sum is a Lyapunov function whose value grows 
exponentially when Z c (t) deviates in both directions from the 
constant Q. This exponential-type Lyapunov function is useful 
for both stabilizing Z c (t) and guaranteeing there is sufficient 
data in Z c (t). Such exponential -type Lyapunov functions are 
previously used in p4) to study the optimal utility-delay 
tradeoff in wireless networks. We define the Lyapunov drift 

A(t)±E[L(H(t+l))-L(H(t))\H(t)], 

where the expectation is with respect to all randomness in the 
system in slot t. 

Define the indicator function l^(t) = 1 if Z c (t) > Q 
and otherwise; define = 1 - Define 6 c (t) = 

V c{t) - Ea:(M c )e£^H(*)- The neXt lemma is P rOVed in 

Appendix |D| 

Lemma 4. The Lyapunov drift A(t) under any control policy 
satisfies 

aw-^E^M*)) 1 #(*)] 

c 

+ Vj2&cE [ptf {t) I H(t)] <S + E Qi c) (t)^ 

nc nc 

J2 Qi c) m E $ (*) + di n (*) - E »«n (*) 1 H(t) 

nc Lb a 

-^D^ml^^-d^^iHit)] 

nc 

-vE E ^M*)) \H(t)} +^E^ E K c) w \ H ® 

c nc 

- W J2 l^tK^w-Q) (e p c (t) 1 H(t)] \ 

c 

+ wJ2 l L c (t)e w ( Q - z °W (E [S c (t) I H(t)] + \ 



(45) 



by induction assumption. 
2) If £>i c) (t) > V6 C , the ORA policy chooses ^{t) = 
dmax and we have 

D ( n c) (t + 1) > i?i c ) (t) - <U > V9 C - d m . dX . 

The proof is complete. 



Appendix C 

We construct a proper Lypuanov drift inequality that leads 
to the UORA policy. Let 

be the vector of all physical and virtual queues in the network. 
Using the parameters w and Q given in the policy, we define 
the Lyapunov function 



where B is a finite constant defined by 

B = \M\ \C\ [(^a'x + rfmax) 2 + (^Ux + /Cx)'] + 

2 \M\ \C\ d 2 m . dx + \C\ [w(2S mdx + e) + e -(— +^) + e ™«] 



By isolating decisions variables in ( [45) , it is not difficult 
to verify that the UORA policy observes the current network 
state H(t) and minimizes the right-hand side of ( p3| l in every 
slot. 

Appendix D 

We establish the Lyapunov drift inequality in ( |45) , Applying 
to ([TJ the fact that [(a— b) + — c] + = (a— b— c) + for nonnegative 
reals a, b, and c, and using Lemma [7] in Appendix [G] we have 



^([Ql c) (t + i)] 2 -[Qi c) W] : 



< B, 



Q- 



Qi c) (t) ( E a*S (*) + 4 C) w - ^l c) w - E (*)) - 

b a 

(46) 
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where B Q = (/j,°^ x + d max ) 2 + (A max + Mmax) 2 is a finite 
constant. From (|23|) and d22b, we get 



D^(t+1) < [D^(t)-^(t)] + +d^(t). 
Similarly, we obtain for queue Dn\t) 

<B D -D( n c \t)(^(t)-di c \t)), (47) 
where Bp — 2d^ ax is a finite constant. 

Lemma 5. Given fixed constants e > and Q > 0, we define 



e/K 



(48) 



Then 



e w(Z c (t+l)-Q) _ e »(Z c (t)-Q) 



e w(Q-Ze(*+l)) „ e t«(Q-Zc(t)) 



< e 1 ^ + we 



w(Q-Z c (t)) 



Sc(t) 



Sc(t) 



(49) 



(50) 



Z c (t+l) < 



Proof: Define 5 c (t) = v c (t) - EoMarflW and <W = 
max(iy mal ,^ al ); we have \S c (t)\ < 5 max and v c {t) < S max . 
Equation ( |27] > yields 

Z c (t)-<J c (i) if Z c (i)> 

^max ~~i~ Mmax 

if Z c (i) < v m . dx . 
Since w > 0, we have 

e toZ e (*+l) < e w(>W+(C) + e wZ c {t) e -wS^t) ^jj 

because the first term is bounded by the second term if Z c (t) < 
t'maxj and is bounded by the third term otherwise. Multiplying 
both sides by exp(— wQ) yields 



w(Z a (t+l)-Q) < _i«(iw+/Cx) 



+ e ifl(2c(t)-Q) 



1 - w<5 c (t) + 



u; 2 <? 



max wSjnax 



, (52) 



where the last term follows the Taylor expansion of e ""w). 
If we have 

^e««- < f , (53) 

then plugging ( f53) ) into ( |52"1 > leads to (H9). Indeed, by definition 
of w in (RE} we get 



2 2 ~ 2 ' 

which uses to < e/<5 2 ax =>• exp(w<5 max ) < exp(e/<5 max ). 
Also, ((27]) leads to 

z c (t + 1) > z c (t) - i/ c (t) +x)/ 2 S,w- 

a 

Define the event _E as 

E:i£ Ql c) (*)>/iSi Vn:(n,d c )G£, 



i.e., all upstream nodes of d c have sufficient data to transmit, 
in which case we must have Ji^ (t) = (t). It follows 

+ > ( Z c (t) - f c (t) if event £ happens (M) 
1 otherwise 

where the second case follows that queue Z c (t) is always non- 
negative. Similar as ( |5Tj ), from |54| we obtain 

e -wZ c (t + l) < 1 + e -wZ c (t) e w6 c (t) 



< 1 + e 



-wZ c (t) 



1 + wS c (t) + 



w 2 5 2 , i (55) 

max g-u;o m 



in which we use the Taylor expansion of e wSc< ' t \ Bounding 
the last term of ( |55] l by ( f53] l and multiplying the inequality by 
exp(tuQ), we obtain <f50]>. ■ 



(56) 



Now, define the indicator function 1^(4) = 1 if Z c [t) > Q 
and otherwise. Define = 1 — 1^ (i). Then for any real 
number a, 

oe i»(z.(t)-0) = a(1 R( t ) + lL( t )) e -(^(t)-Q) 
< |a| + al*(*) e »(^W-0). 
Similarly, we have 

ae «(0-z e {t)) < | o | + aiL( t ) e «(0-^.(t)). (57) 

Using a = -tw(J c (i) - e/2) in ( |56] > and a = tu(<5 c (t) + e/2) 
in ([57]), inequalities ([49]) and ( |50] ) lead to 



s w(Z c (t+l)-Q) _ e w(Z c (t)-Q) < e «>(iw+/CJ 

+ w(6 max + |) - u,l*(t)e , »<^W-«> [S c (t) - | 



(58) 



(59) 



Finally, summing ( |46] i and (J47J over n ^ d c for each c 6 C, 
summing ( |58] l and (J59j over c <G C, and taking conditional 
expectation, we have 



A(t)<B + ^2Ql c \t)\i 



c) 



L 6 



l ^l*( t ) e ^.(t)-0)( E[tfc(t) | H(t)] _|) 



,^lL(t) e -W-^W) 



(E [<5- c (t) 1 fl-(t)] + |) , (60) 



where B is a finite constant defined as 

B = \Af\ \C\ (B Q + B D ) 
+ \C\ \w(2S mz 



„,, N t- e) + e-^+zw) + e »0 



(61) 



The constants Bq and are defined right after (J46j and ( |47| , 
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respectively. Adding to both sides of ( |60| l 

-V^M^t)) \ H(t)] +Vj2VcE[p ( :\t) \ H(t) 

c nc 

yields ( |45| ). 



Appendix E 
Proof of LemmaO 

The boundedness of the queues Q$ (t) and D n c ^ (t) follows 
the proof of Lemma [2] For the queues Z c (t), we again prove 
by induction. Suppose Z c (t) < Z^m, which holds at t = 
because we let Z c (0) — 0. Consider the two cases: 

1) If Z c (t) < Z max - /<C X , then by j27) 

^c(* + l)<^c(t)+AC<^- 

2) If Z c (t) > Z^l - <ax. *en from d32) we obtain 



Q 

\-2<L 



Z&)-Q>Z&-& 

> 0, (62) 

where the last inequality follows our choice of V satis- 



fying V9 C + 2d max > w as mentioned in Section |V-C 
As a result, in ( f2"9] > we have 



where the inequality follows (a) in ( |62| l. Since all queues 
Qn^(i) for n ^ d c are deterministically bounded by 



Qmix, we have Q^(t) > Qn'(t) for all nodes n such 
that (n,d c ) £ C. Consequently, the UORA policy does 
not transmit any class c packets over the links (n, d c ) 
and the virtual queue Z c (t) has no arrivals. Therefore 
Z c (t + 1) < Z c {t) < 
We conclude that Z c (t + 1) < zi ax . The proof is complete. 

Appendix F 
Proof of Theorem|2] 

Let the throughput vector (r*) and the flow variables 
(<Zn^*i fab*) be the optimal solution to the utility maximiza- 
tion (|T2j-(jT3J. We consider the stationary policy that observes 
the current network state H{t) and chooses in every slot: 

1) v i n ) *(t)=di c) *(t)= q k c) *. 



-,(<•) 



2) Mif (t) 



Ac)* 

Jab 



3) v*{t) = r* + f if Z c (t) > Q, and i/*(t) 



otherwise. 

(c)* 



The first part is a feasible allocation because the value of g„ 
is at most A max + /^™ ax , which is less than or equal to d max 
by Assumption [T] The second part is feasible because the link 
rates /Am satisfy the link capacity constraints ((8). The third 
part is feasible because we assume v max > max ce e r* + e/2. 

Under this stationary policy, by using the equality in |9]) 
and using ( fOj l, we have ^ a Av]*(i) — r* in every slot. As a 
result, 



E[S*(t)\Z c (t)>Q] 



E[v* c (t)-Y,Vat(t)\Zc(t)>Q 

a 

E[v*(t)-r* c \Z c (t)>Q} = ^, 



E[5*(t) | Z c (t)<Q}=0. 

The UORA policy minimizes the right-hand side of ( |45| in 
every slot. Using the decisions in the stationary policy, we 
can upper bound (|45]l under the UORA policy by 



A(t)-Vj2®MMt)) \H(t)] + Vj2 ^[p^(t) \H(t) 

c nc 

<b + \c\ ~e w Q -vJ2hc(K(t)) + vJ2e c qi c) * 

c nc 

- E (*) ( E /#* + - W - E fit) 

nc b a 

= B 1 -Vj2hM(t)) + Vj2 () cq { n c) *, (63) 

c nc 

where we define 

B 1 = B+ - \C\wee wQ (64) 

and B is given in ( |6"Tj ). The last equality in ( |63] l is because 
the variables 
constraints |7) 



(c)* ' ' (c)* 

the variables /V and ' satisfy the flow conservation 



The utility functions g c (-) are assumed to have bounded 
derivatives with | | < m c f° r a ll x > 0. Thus, the utility 
functions h c (-) have bounded derivatives with < r nn c + 

9 C for all x > 0. Using <E {r*, r* + e/2}, we have 

e 



M^))>Mr;)-j(ffl«+« c ). 



(65) 



If v*(t) = r* then it is trivial. If z/*(t) = r* + e/2 then by 
mean- value theorem there exists y E (r*, r* + e/2) such that 

W(*)) = M<) + (^W-<K(^ 

>h c (r* c )- e -(m c + 6 c ). 
Using §65\ in ( |63| yields 

A(t)-vj2nh c (Mt)) i +^E^ E K c) (*) i ff w 



c nc 

= B 1 -VY,9c«) + vY,°c^ 



c 

(66) 



where the last equality uses h c {x) — g c (x) — c x and r* = 
" 9n^*) in ( ff3] >. The next lemma is useful. 

Lemma 6. Define v c (t) = mm[Z c (t),is c (t)] as the virtual 
data served in queue Z c (t) in slot t. If Q > i^ max then 
- ^c(*)| < e f° r a H * under the UORA policy. 



Proof of Lemma p 
to the convex program 



The value of u c (t) is the solution 
3D)-§T). If Z c (t) > Q, then we have 
Z c (t) > Vmm. > v c {t) and v c (t) = v c (t). If Z c (t) < Q, the 
problem (pOl-pTj) reduces to 

maximize Vh c (u c (t)) - w v c {t)e w ^ Q ~ z " {t)) (67) 
subject to < v c {t) < v max . (68) 
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For all x > e, we have by mean-value theorem From ( |23| > and f27] > we have 

y hM^M = vK{Xe): for some , e e M) D^(t + 1) > flM(t) - p W(t) + #)(*), 

a; — e 

< < we w t Q - Ze ®\ (69) 



z c (t + 1) = z c (t) - v c (t) + e w. 



where the first inequality results from the choice of C that Taking expectation and time average yields 

yields h'(x) < for all x > e. Rearranging terms in d69l), for — — 

allz> ewe have ^ ^ $\t) >%\ t) -E[pU(t)]/t, (74) 

Vh c {x)-wxe< Q - z ^ <Vh c (e)-wee w ( Q - z °W. W = E^t^ + E [^c(0) - Z c (t)]/t. (75) 



Therefore, the solution to the problem doTb-dMll must satisfy XT , , c a .. £ 

r > — > " — " J Now, by the law of flow conservation, the sum of exogenous 

v c [t) < e. Since < v c (t) < v c (t) < e, we conclude . , . . , , . c , ,. , , . 

i 7 \ — ~ / si ~~ w _ v ' ~ arrival rates must be equal to the sum of delivered throughput, 

\ v c\ ) v c \ )\ _ t. ■ t j me avera g es f dropped packets, and queue growth rates. In 

other words, we have for each class c 6 C and for all slots t 

Now, from Lemma © we have £ *W = £ (i)+E p&j, (t) + i £ E [Q W (t )] . (76) 

M"ci<M?c(*)) + eKl« c ). (70) " . __ ° 

Combining (|74|i-(|76Jl yields 

If v c (t) = v c (t) then this is trivial. Otherwise, if v c (t) < 

v c {t) then by mean- value theorem we obtain for some y G \($ — S~^(pn (t) — v c (£) < 

(v e (t), v c (t)) that 

n n 



h c {v c (t)) = h c {D c (t)) + M«) ~ Mt))h'M ]E [Z c (t) - Z c (0)} + ]J2 E [ D n ] W + QnHt) 

<h c (V c (t)) + e{m c + 9 c ). 



(77) 



Plugging ( f70| > into |66) and rearranging terms yield Finally, using the boundedness of queues Q%>(t), D X n'(t), 

^-^ and Z c (t) in Lemma |3l and the continuity of q c ( ), we obtain 

A(t)-Vj2E[h c (v c (t))\H(t)] from ^ and |77| thi 



+ y^fl c E^(t) | H{t)\ g e (v c (tfj = Sc( t Hm£^(t)) = g c (r c ), (78) 

nc a 

<B 1 + ^j2(^+^- v Y,9^)+yJ2^ xi n c) ■ &2o ( E A » } - E ^ - w) < o, (79) 

c c nc ^ n n ' 

where the last equality of ( f78] ) uses the definition in ( |25] l. 
Define the time average Taking a limit of ( |73"j ) ast^cxj and using ( |78j ) and ( |79| i, we 



t _ 1 obtain 



Define ^(t), c4 c) (t), and A^C*) similarly. Taking expecta- where the constant B 1; defined in |64|, is 
tion and time average over r G {0, t— 1} in fTTli, dividing n a i \ r \ r/,,out , . \2 , / A , ,,in -.2 , ,2 i 

, T/ . f j i * t . • ... t & i>l = l-A/ | C [(Mmax + «max) + (Anax + Mmax) + 2 «max 

by I/, rearranging terms, and applying Jensen s inequality to . 

the functions h c (-), we get ' " + \C\ [w(25 m , x + e) + e w ^+^ + —e w ® + e w ® 

(80) 



£ h c (v&)) + E ®c - ^ (tj) > 



The proof is complete. 



EfcW) * Dm . + W . (72) 



Appendix G 



Adding and subtracting y2 n 9 c v c (t) at the left-hand side T _ r . S1 . 

r ftti j j £ r i / \ ■ u Lemma 7. If a queue process \ Q(t)\ satisfies 

of {12) and using the denmtion of n c (-) yield n r ,J 

, n Q(t + i)<(Q(t)-^)) + + «W, (81) 

(t) — v c (t) > where a(t) and are nonnegative bounded random variables 

c ™ " ' with a(t) < a max and 6(f) < 6 max , then there exists a positive 

> e aM - f - E[L( T y o))] - 1 E(- c + »c). c ° nstant s such ^ 



(73) kQ 2 (t + 1) - Q 2 W) < B - Q(*) (&(*) - «(*))• 
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Proof of Lemma [7} Squaring both sides of pT| ) yields 

Q 2 (t + 1) < (Q(t)-b{t)) 2 + a 2 (t) + 2Q(t)a(t) 
<Q 2 (t)+B-2Q(t) (b(t)-a(t)) 



where B is a finite constant satisfying B > a^ ax + fe max . 
Dividing the above by two completes the proof. 



